Data processing apparatus and method of controlling same

ABSTRACT

An apparatus comprises supply means for changing an amount of light sinusoidally and supplying to a projection means a patterned light signal for sequentially projecting on an object to be measured at least three patterns of patterned light having phase differences; obtainment means for obtaining image data acquired by capturing the object; first generation means for generating a phase image from image data acquired by capturing the object on which the patterned light is sequentially projected; second generation means for generating, based on the phase image, a shape image which indicates a distance between a surface of the object and a viewpoint in the capturing; and correction means for calculating an amount of correction for the shape image from a correspondence between a pixel value of a high-frequency shape image indicating a high-frequency component of the shape image and a pixel value of the phase image.

TECHNICAL FIELD

The present invention relates to data processing at a time at which ashape of an object is measured.

BACKGROUND ART

A method for projecting a striped pattern in which an amount of lightperiodically changes on an object to be measured (hereinafter referredto as a target), image-capturing the target, analyzing the stripedpattern of the top of the image acquired by the capture, and measuring athree-dimensional shape of the target (a distance to the target) isknown.

The measurement method called the phase shift method projects a stripedpattern for which the amount of light sinusoidally changes (hereinafterreferred to as a sinusoidal pattern) on a target while changing thephase a plurality of times, and captures the target. For example, itobtains three kinds of images by capturing the target after projecting asinusoidal pattern of a phase shift of 2π/3 and a sinusoidal pattern ofa phase shift of 4π/3 in relation to a first sinusoidal pattern. At thattime, the phase of each pixel is acquired by processing for calculating,by an arc tangent, pixel values (luminance values) of the same pixelpositions in each obtained image. Since the phase corresponds to aprojection angle for the projector, the three-dimensional shape of thetarget can be measured from a triangulation method principle if theposition relationship of the projector and image capturing device areknown beforehand.

However, for the phase shift method, if there are nonlinearities in anoutput characteristic of the projector and the image capturing device,projecting/capturing a correct sinusoidal pattern is not possible and anerror arises in a phase estimated for each pixel. The result of this isthat a stripe error arises in the measurement results of thethree-dimensional shape.

Japanese Patent Laid-Open No. 2009-210509 discloses a measurementapparatus which is not dependent on output characteristics of aprojector and image capturing device by modulating/combining sinusoidalpatterns. S. Zhang. P. S. Huang “Phase error compensation for a 3-Dshape measurement system based on the phase-shifting method” OpticalEngineering 46(6) dated June, 2007 discloses a correction table whichcorrects phase errors caused by nonlinearities of output characteristicsof a projector and an image capturing device by using a white referenceplate of uniform reflectance. Also, T. Zaman “Development of aTopographic Imaging Device” Delft University of Technology dated March,2013 discloses a measurement method called the stereo phase shift methodwhose object is to perform a measurement independent of nonlinearitiesof output characteristics of a projector and image capturing devices byarranging two image capturing devices.

According to the techniques disclosed in the above described documents,suppressing measurement errors due to nonlinearities of outputcharacteristics of a projector and image capturing device is possible.However, causes of an occurrence of measurement errors are not limitedto just the nonlinearities of the output characteristics of theprojector and the image capturing device. In particular, measurementerrors arise due to light intensity fluctuations caused by a lightemission timing of the projector or an image capturing timing of theimage capturing device in a case when a generic projector and camera areused as the projector and the image capturing device. Regarding suchfluctuation due to the emission/image capturing timing, in actuality,fluctuation at a time of measuring the target is unknown and appropriatecorrection is difficult even if characteristics of the measurementapparatus are obtained by using a white reference plate or the likebeforehand.

SUMMARY OF INVENTION

A first embodiment of the present invention suppresses a measurementerror caused by unknown light intensity fluctuations in a measurement ofa shape of an object.

The first embodiment of the present invention comprises the followingconfiguration.

A data processing apparatus, comprising: supply means for changing anamount of light sinusoidally and supplying to projection means apatterned light signal for sequentially projecting on an object to bemeasured at least three patterns of patterned light having phasedifferences; obtainment means for obtaining image data acquired bycapturing the object; first generation means for generating a phaseimage from image data acquired by capturing the object on which thepatterned light is sequentially projected; second generation means forgenerating, based on the phase image, a shape image which indicates adistance between a surface of the object and a viewpoint in thecapturing; and correction means for calculating an amount of correctionfor the shape image from a correspondence between a pixel value of ahigh-frequency shape image indicating a high-frequency component of theshape image and a pixel value of the phase image.

By virtue of exemplary embodiments of the present invention, it ispossible to suppress a measurement error caused by an unknown lightintensity fluctuation by a correction based on data acquired by ameasurement in an object shape measurement.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the presentinvention and, together with the description, serve to describe theprinciples of the invention.

FIG. 1 is a block diagram illustrating an example configuration of anapparatus for measurement of a shape of embodiments.

FIG. 2 is a flowchart for describing processing of a data processingunit.

FIG. 3 is a flowchart for describing processing of a shape imagegeneration unit.

FIG. 4A is a view for illustrating an example of code images.

FIG. 4B is a view for illustrating an example of code images.

FIG. 5A is a view for describing phase unwrapping.

FIG. 5B is a view for describing phase unwrapping.

FIG. 6 is a view for describing an epipolar line.

FIG. 7A is a flowchart for describing processing of a correction unit.

FIG. 7B is a dataflow chart illustrating a dataflow in the correctionunit.

FIG. 8A is a view for describing a generation of a correction function.

FIG. 8B is a view for describing a generation of a correction function.

FIG. 8C is a view for describing a generation of a correction function.

FIG. 9 is a flowchart for describing processing of a correction unit ofa second embodiment.

FIG. 10A is a view for illustrating examples of φ-H tables generated forfour divided areas.

FIG. 10B is a view for illustrating examples of φ-H tables generated forfour divided areas.

FIG. 10C is a view for illustrating examples of φ-H tables generated forfour divided areas.

FIG. 10D is a view for illustrating examples of φ-H tables generated forfour divided areas.

DESCRIPTION OF EMBODIMENTS

A data processing apparatus, a data processing method, and a measurementapparatus of embodiments according to the present invention will bedescribed hereinafter in detail, with reference to the drawings. Notethat these embodiments do not limit the present invention according tothe scope of the claims, and not all of combinations of configurationsdescribed in the embodiments are necessarily required with respect tothe problem solving means according to the present invention.

First Embodiment

Hereinafter, description will be given of an example that applies thepresent invention to a measurement apparatus of a stereo phase shiftmethod which uses one projector and two image capturing devices.However, the present invention can be applied even in relation to aphase shift method which uses one image capturing device.

Apparatus Configuration

An example configuration of a shape measurement apparatus of embodimentsis illustrated by a block diagram of FIG. 1. The measurement apparatuscomprises a projector 13 which projects a striped pattern light on anobject to be measured (hereinafter referred to as a target) 12 as wellas a first image capturing device 14 a and a second image capturingdevice 14 b which capture an image of the target 12 onto which thestriped pattern light is projected. Also, a data processing unit 11performs various calculations and control of the projection of thestriped pattern light and capture of the measurement apparatus, andoutputs shape information of the target 12 after processing the capturedimage data.

In the data processing unit 11, a pattern supply unit 101 generatespattern data which becomes the basis of the striped pattern light basedon a pattern shape program and a program which sets a projection time orthe like which are stored in a ROM, a flash memory, or the like.Alternatively, the pattern supply unit 101 may, instead of generatingthe pattern data, store a plurality of types of pattern data generatedin advance in the ROM or the flash memory of the like and sequentiallyselect the pattern data. In the description below, an example usingthree kinds of sinusoidal patterns and four kinds of code patterns asthe pattern data will be described. The sinusoidal patterns aremonochrome data and the amount of light in the horizontal directionperiodically changes sinusoidally, and each sinusoidal pattern has aphase difference of 2π/3. The details of the code patterns are describedlater.

The pattern supply unit 101 supplies a patterned light signal based onthe pattern data generated or selected to the projector 13 via aninterface 109 such as a USB interface. The projector 13 sequentiallyprojects the code pattern light and the sinusoidal pattern light, whichare striped pattern light respectively based on the three kinds ofsinusoidal pattern light signals and the four kinds of code patternlight signals, onto the target 12 at respective predetermined projectiontimes. The first image capturing device 14 a sequentially captures thetarget 12 on which the sinusoidal pattern light or the code patternlight is projected from a first viewpoint. The second image capturingdevice 14 b sequentially captures the target 12 on which the sinusoidalpattern light or the code pattern light is projected from a secondviewpoint. Each image capturing device performs a seven pass capturecorresponding to the three kinds of sinusoidal pattern light and thefour kinds of code pattern light from different viewpoints, and thecaptured image that the two the image capturing devices 14 a and 14 boutput is a total of 14 passes.

The image data that the image capturing devices 14 a and 14 b capturesis temporarily stored in an image memory 102 via the interface 109. Siximage data items (hereinafter referred to as sinusoidal images) of thetarget 12 on which the sinusoidal pattern light is projected, which arestored in the image memory 102, are output to a phase image generationunit 103. Also, eight image data items (hereinafter referred to as codeimages) of the target 12 on which the code pattern light is projected,which are stored in the image memory 102, are output to a shape imagegeneration unit 104.

The phase image generation unit 103, although explained later in detail,generates a first phase image and a second phase image from the sixsinusoidal images, outputs these phase images to the shape imagegeneration unit 104, and outputs the first or second phase image to acoordinate transformation unit 106. The shape image generation unit 104,although explained later in detail, calculates a shape image of thetarget 12 based on the first and second phase images, the eight codeimages, and a camera parameter obtained from a camera parameter memory105, and outputs the shape image to a correction unit 107. Note, theshape image is data which indicates a distance between the surface ofthe target 12 from the first viewpoint to each pixel for example, andthere are also cases in which it is called a “range image”.

The coordinate transformation unit 106 performs a coordinatetransformation (rotation) such that the optical axis direction of thefirst or second phase image matches with the Z axis (depth direction) ofworld coordinates based on the camera parameter obtained from the cameraparameter memory 105. The coordinate transformation unit 106 outputs thecoordinate transformed phase image (hereinafter referred to as anormalized phase image) to the correction unit 107. The worldcoordinates are a coordinate system for the time at which the shapeimage generation unit 104 calculates the shape image, and the originpoint of the world coordinates corresponds to the viewpoint of the shapeimage.

The correction unit 107, although explained later in detail, correctsthe shape image based on the shape image and the normalized phase image,and outputs the corrected shape image (hereinafter referred to as acorrected shape image) to an output unit 108. The output unit 108 is forexample a USB interface which outputs data of the corrected shape imageto an external display apparatus or image processing apparatus.

Note, a function of the data processing unit 11 can be realized bysupplying software for shape measurement to a computer device and by thecomputer device executing the software.

Measurement Processing

Processing of the data processing unit 11 will be described by theflowchart of FIG. 2. The pattern supply unit 101 supplies a patternedlight signal to the projector 13 in order to project a striped patternlight on the target 12 (step S201). The interface 109 inputs image dataof the target 12 on which the striped pattern light is projected that isacquired by the capturing of the image capturing devices 14 a and 14 band stores the image data to the image memory 102 (step S202). The dataprocessing unit 11 determines whether or not obtainment of the capturedimage of the three kinds of sinusoidal pattern lights and the four kindsof code pattern lights (14 passes total) is finished (step S203), and ina case of incompletion, it returns the processing to step S201 andchanges the striped pattern light.

When obtainment of the captured images finishes, the phase imagegeneration unit 103 generates the first and second phase images from thesinusoidal images inputted from the image memory 102 (step S204). Theshape image generation unit 104 generates the shape image of the target12 based on the code images inputted from the image memory 102, thefirst and second phase images, and the camera parameter obtained fromthe camera parameter memory 105 (step S205).

The coordinate transformation unit 106 generates a normalized phaseimage for which coordinates of the first (or second) phase image aretransformed from the coordinates of the first image capturing device 14a (or the second image capturing device 14 b) to world coordinates (stepS206) based on the camera parameter obtained from the camera parametermemory 105. The correction unit 107 generates a corrected shape imagewhich corrects the shape image based on the shape image and thenormalized phase image (step S207). The output unit 108 outputs thecorrected shape image to an external display apparatus or imageprocessing apparatus (step S208). This completes the shape measurementprocessing sequence.

Phase Image Generation Unit

The phase image generation unit 103 generates a first phase image fromthe three sinusoidal images acquired by capturing from the firstviewpoint by the first image capturing device 14 a and generates asecond phase image from the three sinusoidal images acquired bycapturing from the second viewpoint by the second image capturing device14 b.

A pixel position of the image is (x, y), and the luminance value of eachpixel of the three sinusoidal images is I1(x, y), I2(x, y), and I3(x,y). Also, for a three element vector b=(b1, b2, b3)^(T), b1=I1(x, y),b2=I2(x, y), and b3=I3(x, y). Here, T is a symbol which represents atransposition. Also, regarding a 3×3 matrix A, each element of thematrix A is denoted a_(ji). j and i are indexes representing row andcolumn. Here, each element of the matrix A is given as follows.

a ₁₁=sin(0), a ₁₂=sin(2π/3), a ₃=sin(4π/3)

a ₂₁=cos(0), a ₂₂=cos(2π/3), a ₂₃=cos(4π/3)

a ₃₁=1, a ₃₂=1, a ₃₃=1

Next, vector p=(p1, p2, p3)^(T) will be calculated from the followingequation by using matrix A and vector b.

p=A ⁻¹ b  (1)

Here, A⁻¹ is an inverse matrix of A.

Then, phase φ will be calculated from the following equation.

φ=tan⁻¹(−p1/p2)  (2)

-   -   Here, tan⁻¹ is arc tangent and φ, which satisfies tan φ=p1/p2,        is returned, the domain of φ is [0, 2π].

By the above described calculation, phase φ(x, y) of each pixel isacquired and φ(x, y) is a “phase image”.

Shape Image Generation Unit

Processing of the shape image generation unit 104 will be described bythe flowchart of FIG. 3. The shape image generation unit 104 extracts acorresponding point where the pixel values from the first phase imageand the second phase image are the same as each other, and generates ashape image which indicates the shape of the target 12 by calculatingthree-dimensional coordinates of the corresponding point by theprinciples of the triangulation method.

The shape image generation unit 104 inputs the first and second phaseimages and the code images (step S301). The shape image generation unit104 determines a relationship between the period of the sinusoidalpattern and the pixel position (x, y) of the first phase image based onthe four code images acquired by the capturing from the first viewpoint(step S302). Similarly, the shape image generation unit 104 determines arelationship between the period of the sinusoidal pattern and the pixelposition (x, y) of the second phase image based on the four code imagesacquired by capturing from the second viewpoint (step S303). Note, thedetermination of the relationship between the pixel position and theperiod is something in which it is determined what number sinusoidalpattern period the pixel position corresponds to. Also, images for whichfour kinds of patterns called Gray codes are projected/captured are usedas the code image.

FIG. 4A and FIG. 4B are views for illustrating an example of codeimages. The Gray code illustrated in FIG. 4A is a pattern having lightareas and dark areas comprising light portions and dark portions, andthe light and dark regions in the horizontal direction periodicallychange similarly to the sinusoidal patterns. In the embodiment,light/dark patterns 401 to 404 of the four passes illustrated in FIG. 4are used. When the dark portions (black areas) and light portions (whiteareas) of each image for which a pattern having light areas and darkareas is projected/captured are encoded as “1” and as “0” respectively,4 bit binary values are acquired and 16 areas become distinguishable.Consequently, the relationship between the pixel position (x, y) and theperiod can be determined by causing the size of each area of the Graycode to align with one period of the sinusoidal pattern if a period M ofthe sinusoidal pattern is equal to or less than 16 (M<16).

An image other than the Gray codes may be used as a code image. Forexample, a pattern having light areas and dark areas which is called abinary pattern illustrated in FIG. 4B may be used. Also, the types ofpattern are not limited to four types, and it is possible to increase ordecrease the number to conform to the size or the like of the target 12.

Next, the shape image generation unit 104 performs processing which iscalled “phase unwrapping” in order to extract a corresponding pointwhere the pixel values from the first phase image and the second phaseimage are the same as each other based on the determination result ofthe relationship between the pixel position and the period (step S304).FIG. 5A and FIG. 5B are views for describing phase unwrappingprocessing. Because the pixel values of the phase image changeperiodically within the range [0, 2π], pixels having the same pixelvalue (phase) appear repeatedly in a horizontal direction as illustratedin FIG. 5A, and there is not a single unique corresponding point atwhich pixel values are the same. The phase unwrapping processing isprocessing for adding 2πm to the pixel value of an m-th (m=0 to M−1)period of the phase image that an M-period sinusoidal wave includes. Byphase unwrapping, the periodicity of the phase image as illustrated inFIG. 5B is removed and it becomes possible to extract a correspondingpoint where the pixel value (phase) is the same.

Next, the shape image generation unit 104 obtains the camera parameterfrom the camera parameter memory 105 (step S305). In the embodiment, abasis matrix F and perspective projection matrices P₁ and P₂ areobtained as the camera parameter. Then, the shape image generation unit104 extracts the corresponding point where pixel values from the firstphase image and the second phase image are the same as each other (stepS306). The corresponding point is retrieved from the top of an epipolarline. The corresponding point can be uniquely determined even in a casewhen a sinusoidal pattern which changes in the horizontal direction isused by limiting a search for the corresponding point to the epipolarline. In a case in which the positions and orientations of the two imagecapturing devices 14 a and 14 b are known beforehand, the epipolar lineis a constraint relating to a corresponding point when one point in athree-dimensional space is captured respectively. Note, thecorresponding point may be calculated by a full search although it takesmore calculation time.

FIG. 6 is a view for describing an epipolar line. Regarding a linesegment which connects a viewpoint position C1 to a point X inthree-dimensional space and a line segment which connects the viewpointposition C1 to a viewpoint position C2, a flat surface (epipolarsurface) in three-dimensional space which includes the two line segmentsis uniquely determined. A straight line which corresponds to anintersection of the epipolar flat surface and an image capturing surfaceΠ2 is an epipolar line. It is known that an epipolar line is representedby an equation x₁ ^(T)Fx₂=0 using the basis matrix F which includesinformation which relates to the positions, orientations, and internalparameters of the two image capturing devices 14 a and 14 b. Here, x₁and x₂ represent pixel positions in homogeneous coordinates for eachcamera coordinate system. For example, a corresponding point of capturepoint x₁ (corresponding to a pixel) of point X in an image capturingsurface Π1 is retrieved on the epipolar line of the image capturingsurface Π2 and capture points x₁ and x₂ are extracted as correspondingpoints in FIG. 6.

Next, the shape image generation unit 104 calculates a distance betweenan origin point of the world coordinates and the surface of a target 12based on the capture points x₁ and x₂ by using the principles of thetriangulation method (step S307). It is known that the distance from theorigin point of the world coordinates to point X of the surface of thetarget 12 can be calculated from relationships of wx₁=P₁X and wx₂=P₂X byusing the perspective projection matrices P₁ and P₂ which includeinformation relating to the positions, orientations, and internalparameters of each image capturing device. Then, the shape imagegeneration unit 104 generates data of the shape image which indicatesthe calculated distance of the surface of the target 12 (step S308).

Correction Unit

The correction unit 107 calculates a correspondence of shape and phasefrom the shape image and the normalized phase image, estimates an errorcomponent of the shape image based on the correspondence, and performscorrection of the shape image by the error component. Processing of thecorrection unit 107 will be described by the flowchart of FIG. 7A. Also,a dataflow in the correction unit 107 is illustrated by a dataflow chartof FIG. 7B.

The correction unit 107 inputs the shape image S from the shape imagegeneration unit 104 (step S701) and obtains a high-frequency componentof the shape image S by high-pass filter processing (HPF processing)(step S702). In the extraction of the high-frequency component, it isnecessary to extract a frequency component such that a stripe shapeerror component which arises in the shape image S before correction isincluded, although any other method may be used. Hereinafter, thehigh-frequency component of the shape image S is called “ahigh-frequency shape image H”.

Next, the correction unit 107 inputs the normalized phase image q fromthe coordinate transformation unit 106 (step S703). The normalized phaseimage c is an image that is coordinate transformed (rotated) so that theoptical axis direction of the first (or second) phase image and the Zaxis (depth direction) of the world coordinate match. The correctionunit 107 generates a φ-H table which indicates the correspondencebetween the phase value of the normalized phase image q and the pixelvalue of the high-frequency shape image H (step S704). In other words, apixel value φ (x, y) of the normalized phase image and a pixel value H(x, y) of the high-frequency shape image are obtained for each pixelposition (x, y). Then, the pixel values H corresponding to the phasevalues φ of the range [0, 2π] are tallied and a φ-H table (FIG. 8A)which plots an average value of the tallied pixel values H in relationto the phase value φ is generated.

Next, the correction unit 107 uses the φ-H table to derive a correctionfunction fc for the shape correction (step S705). FIG. 8A, FIG. 8B, andFIG. 8C are views for describing generation of the correction functionfc. The correction unit 107 derives, as the correction function fc, afunction indicated in FIG. 8B that approximates a curve that the φ-Htable, which is illustrated in FIG. 8A, indicates. In the embodiment, upto the fourth-order term of a Fourier series development is used as thecorrection function fc. Specifically, an approximation that accords to asinusoidal wave is performed.

Next, the correction unit 107 uses the correction function fc tocalculate a corrected shape image C which corrects the shape image S(step S706). Specifically, the pixel value φ(x, y) of the normalizedphase image is obtained and an amount of correction f(φ(x, y)) whichcorresponds to the error component according to the correction functionfc is calculated for the pixel position (x, y). Then, the amount ofcorrection f(φ(x, y)) is subtracted from the pixel value S(x, y) of theshape image and the corrected shape image C, which is expressed by thepixel value C(x, y)=S(x, y)−f(φ(x, y)), is generated.

As described above, a measurement error arises from light intensityfluctuation due to the light emission timing of the projector or theimage capturing timing of the image capturing device if a genericprojector and camera are used as the projector and the image capturingdevice in accordance with the stereo phase shift method. Regarding suchfluctuation due to the emission/measurement timing, in actuality,fluctuation at a time of measuring the target is unknown and appropriatecorrection is difficult even if characteristics of the measurementapparatus are obtained by using a white reference plate or the likebeforehand. Meanwhile, estimating an appropriate amount of correctionfrom only the data obtained in a measurement by focusing on therelationship of a light intensity fluctuation and measurement error ispossible by virtue of the shape image correction processing in theembodiment.

A sinusoidal phase error in relation to a true phase is, as illustratedFIG. 8C, included in the phase that the phase image generation unit 103calculated if a slight light intensity fluctuation occurs as therelationship between the light intensity fluctuation and the measurementerror. A stripe shape error component occurs in the shape image S beforecorrection due to the phase error. Regarding the sinusoidal wave,characteristics cannot be obtained beforehand because the phase oramplitude in accordance with the light intensity fluctuation isdifferent.

Meanwhile, the high-frequency shape image H in the correction unit 107is an image in which the stripe shape error component and thehigh-frequency component that the true shape has are mixed. Anappropriate shape correction is something in which only the stripedshape error component is removed from the shape image S while thehigh-frequency component that the true shape has is retained. In otherwords, the present embodiment focuses on the characteristics of theerror illustrated in FIG. 8C, and only the stripe shape error componentf(φ(x, y)) is separated by extracting components of the sinusoidal wavefrom the relationship of the phase and the high-frequency shape.Consequently, it is possible to acquire a post-correction shape image Cin which only the stripe shape error component f(φ(x, y)) is removedwhile retaining the high-frequency component that the true shape has.

In this way, in an apparatus for measurement of the stereo phase shiftmethod which uses one projector and two image capturing devices, it ispossible to use only data obtained in the measurement to perform anappropriate correction in the measurement result, and suppress themeasurement error even if the light intensity fluctuation in themeasurement of the shape is unknown.

Variation

In the above description, although an example of a measurement apparatuswhich uses two image capturing devices is shown, three or more imagecapturing devices may be used. Also, the present invention can beapplied even in relation to a phase shift method which uses one imagecapturing device. In this case, it is advantageous to use a techniquefor reducing error caused in a nonlinear output characteristic of aprojector and an image capturing device that the above prior artdocument discloses. In other words, a correction table which correctsphase error caused by nonlinearity of output characteristics of theprojector and the image capturing device by using a white referenceplate of uniform reflectance is employed.

In the description above, although explanation is given of obtainedinformation only being the shape, color information of the target 12 mayalso be obtained at the same time. In this case, it is advantageous forthe pattern generation unit (or the pattern memory) 101 to provide asolid pattern for uniformly illuminating the target 12. Also, althoughthree kinds of sinusoidal patterns used in the phase calculation wereassumed, three or more kinds may be used. In this case, a pseudo-inversematrix of A as matrix A−1 is used. The phase difference of each patternis arbitrary and it is not necessary that the phase differences beuniform.

Although an example which uses the basis matrix F and the perspectiveprojection matrices P₁ and P₂ as the camera parameter was given in thedescription above, the camera parameter is not limited, and may indicatethe relationship of the position/orientation between cameras. Also,generating the camera parameter by measuring the shape of a known objectrather than retaining the camera parameter is possible.

Also, although using a sinusoidal wave as described above as thecorrection function fc is ideal, an arbitrary function may be used, forexample, a polynomial function may be used. Also, a Fourier seriesdevelopment or a polynomial order can be arbitrarily selected.Additionally, configuration may be such that a plurality of functionsare used to perform the approximation and the function whoseapproximation precision is the best is applied to the shape correction.For example, a φ-H table approximation function square error sum can beused as the approximation precision.

Second Embodiment

Hereinafter, description of a measurement apparatus and a measurementmethod of a second embodiment according to the present invention isgiven. Note that in the second embodiment, there are cases in which thesame reference numerals are added and detailed description is omittedfor configurations approximately the same as those in the firstembodiment.

In the first embodiment, description was given of an example in whichthe normalized phase image (and the high-frequency shape image H areused to generate a φ-H table of the phase value c and the pixel value H(high-frequency component) and a correction function fc is generated.However, there are cases in which the projection pattern cannot beappropriately obtained due to the brightness of the target 12 and theexposure conditions of the image capturing devices 14 a and 14 b, and anarea in which a lot of noise is included in the obtained phase image isgenerated. In these cases, there is a possibility that due to aninfluence of the noise of the phase image and the φ-H table that isgenerated, it will not be possible to successfully extract the shapeerror component.

The correction unit 107 of the second embodiment divides an image intoareas, and derives the correction function fc by using the divided areafor which noise influence is considered to be the minimum. Processing ofthe correction unit 107 of the second embodiment by the flowchart ofFIG. 9 will be described. Note, the processing in steps S701-S703 andstep S706 is the same as the processing of the first embodimentillustrated in FIG. 7A and FIG. 7B, and a detailed description isomitted.

In the generation of the φ-H table, the correction unit 107 splits thenormalized phase image φ and the high-frequency shape image H bothvertically and horizontally for example to make four areas (an upperleft area, an upper right area, a lower left area, and a lower rightarea) (step S704 a). Then, for each divided area, the pixel value ((x,y) of the normalized phase image and the pixel value H (x, y) of thehigh-frequency shape image for the pixel position (x, y) are obtainedand the φ-H table which maps an average value of H (x, y) for φ(x, y) isgenerated (step S704 b). FIG. 10A, FIG. 10B, FIG. 10C, and FIG. 10D areviews for illustrating examples of the φ-H tables generated for the fourdivided areas.

In the derivation of the correction function fc, the correction unit 107derives (step S705 a) functions fc1 to fc4 which approximate curves thatthe four φ-H tables illustrated in FIG. 10A, FIG. 10B, FIG. 10C, andFIG. 10D indicate, evaluates the approximation precision of eachfunction, and the function whose the approximation precision is best isapplied as the correction function fc (step S705 b). For example, a φ-Htable approximation function square error sum may be used as theapproximation precision.

In this way, it is possible to suppress a measurement error by derivingan appropriate correction function fc even if an area which includes alarge amount of noise is generated in the obtained phase image. Note,although an example in which four divided areas are made by bisecting animage both vertically and horizontally was described above, the methodof area division and the number of divided areas are arbitrary, and thedivided areas may overlap other divided areas. Also, the size and shapeof the divided areas can be arbitrarily set and each divided area may bedifferent.

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD™)),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2015-252404, filed Dec. 24, 2015, which is hereby incorporated byreference herein in its entirety.

1. A data processing apparatus, comprising: an obtainment unitconfigured to obtain data representing a plurality of images eachacquired by image-capturing an object onto which a pattern of patternedlight, of which amount of light periodically changes along apredetermined direction, and a plurality of patterns of patterned light,which have phase differences with respect to the pattern of patternedlight, are projected; a first generation unit configured to generate,based on the data representing the plurality of images, datarepresenting a phase image in which each pixel has a phase of periodicchange in pixel value between images of the plurality of images; asecond generation unit configured to generate, based on the datarepresenting the phase image, data representing a shape image in whicheach pixel has a value representing a shape of a surface of the object;and a correction unit configured to calculate an amount of correctionfor correcting the shape image based on the correspondence between ahigh-frequency component of a pixel value of the shape image and a pixelvalue of the phase image. 2-4. (canceled)
 5. The data processingapparatus according to claim 1, wherein the correction unit isconfigured to obtain the high-frequency component by performinghigh-pass filter processing of the shape image.
 6. The data processingapparatus according to claim 1, wherein the correction unit isconfigured to obtain a first pixel value of the phase image and a secondpixel value of a high-frequency shape image in which each pixel has therespective high-frequency component for each pixel position, tally thesecond pixel value corresponding to the first pixel value, and obtain asthe correspondence, a relationship between the first pixel value and anaverage value of the tallied second pixel value.
 7. The data processingapparatus according to claim 6, wherein the correction unit isconfigured to derive a correction function that approximates a curvethat the correspondence indicates.
 8. The data processing apparatusaccording to claim 6, wherein the correction unit is configured todivide the high-frequency shape image and the phase image into areas,obtains the correspondence in each divided area, derives functions thatapproximate curves that these correspondences indicate, and employs afunction for which an approximation precision is best as a correctionfunction.
 9. The data processing apparatus according to claim 7, whereinthe correction unit is configured to calculate the amount of correctionaccording to a pixel value of the phase image and the correctionfunction.
 10. The data processing apparatus according to claim 1,wherein the correction unit is configured to generate a corrected shapeimage resulting from subtracting the amount of correction from the shapeimage.
 11. The data processing apparatus according to claim 1, furthercomprising: the projection unit configured to project the patternedlight on the object; and an image capture unit configured to generatethe data representing the plurality of images.
 12. A method ofprocessing data, the method comprising: obtaining data representing aplurality of images each acquired by image-capturing an object ontowhich a pattern of patterned light, of which amount of lightperiodically changes along a predetermined direction, and a plurality ofpatterns of patterned light, which have phase differences with respectto the pattern of patterned light, are projected; generating, based onthe data representing the plurality of images, data representing a phaseimage in which each pixel has a phase of periodic change in pixel valuebetween images of the plurality of images; generating, based on the datarepresenting the phase image, data representing a shape image in whicheach pixel has a value representing a shape of a surface of the object;and calculating an amount of correction for correcting the shape imagebased on the correspondence between a high-frequency component of apixel value of the shape image and a pixel value of the phase image. 13.(canceled)
 14. A non-transitory computer-readable storage medium storinga computer program that realizes on a computer: an obtainment functionof obtaining data representing a plurality of images each acquired byimage-capturing an object onto which a pattern of patterned light, ofwhich amount of light periodically changes along a predetermineddirection, and a plurality of patterns of patterned light, which havephase differences with respect to the pattern of patterned light, areprojected; a first generation function of generating, based on the datarepresenting the plurality of images, data representing a phase image inwhich each pixel has a phase of periodic change in pixel value betweenimages of the plurality of images; a second generation function ofgenerating, based on the data representing the phase image, datarepresenting a shape image in which each pixel has a value representinga shape of a surface of the object; and a correction function ofcalculating an amount of correction for correcting the shape image basedon the correspondence between a high-frequency component of a pixelvalue of the shape image and a pixel value of the phase image.
 15. Thedata processing apparatus according to claim 1, wherein the amount oflight in the patterned light changes sinusoidally along thepredetermined direction.
 16. The data processing apparatus according toclaim 1, wherein the shape image indicates a distance between a surfaceof the object and a viewpoint in the image-capturing for obtaining thedata representing the plurality of images.
 17. The data processingapparatus according to claim 1, further comprising a supply unitconfigured to supply, to projection unit, a signal for projecting thepattern of patterned light, of which amount of light periodicallychanges along the predetermined direction, and the plurality of patternsof patterned light, which have phase differences with respect to thepattern of patterned light, onto the object.
 18. The data processingapparatus according to claim 1, wherein the obtainment unit isconfigured to obtain the data representing the plurality of imagesacquired by image-capturing, from each of a first viewpoint and a secondviewpoint, the object onto which the pattern of patterned light, ofwhich amount of light periodically changes along the predetermineddirection, and the plurality of patterns of patterned light, which havephase differences with respect to the pattern of patterned light, areprojected, wherein the first generation unit is configured to generatedata representing a first phase image based on data representing aplurality of images acquired by image-capturing the object from thefirst viewpoint and to generate data representing a second phase imagebased on data representing a plurality of images acquired byimage-capturing the object from the second viewpoint, wherein the secondgeneration unit is configured to extract a correspondence point where apixel value of the first phase image represented by the datarepresenting the first phase image matches a pixel value of the secondphase image represented by the data representing the second phase imageand to generate the data representing the shape image based on theextracted correspondence point, and wherein the correction unit isconfigured to calculate the amount of correction for correcting theshape image based on the correspondence between a high-frequencycomponent of a pixel value of the shape image and a pixel value of thefirst phase image or the second phase image.